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EXECUTIVE SUMMARY 



Middle School Mathematics Professional Development Impact Study 

This report presents interim results from the Middle School Mathematics Professional 
Development Impact Study, which is sponsored by the Institute of Education Sciences (IES). The 
report presents results immediately following 1 year of the study’s professional development. 

A future report will present results following 2 years of professional development. 

Student achievement in mathematics has been a focal concern in the United States for many 
years. The National Research Council’s 2001 report and the recent report of the National 
Mathematics Advisory Panel (2008) both called attention to student achievement in mathematics, 
and both called for all students to learn algebra by the end of eighth grade. Reports have argued, 
further, that achieving this goal requires that students first successfully learn several topics in rational 
numbers — fractions, decimals, ratio, rate, proportion, and percent. These topics are typically covered 
in grades 4 through 7, yet many students continue to struggle with them beyond the seventh grade. 
The National Mathematics Advisory Panel wrote that “difficulty with fractions (including decimals 
and percent) is pervasive and is a major obstacle to further progress in mathematics, including 
algebra” (p. xix). The panel also specified that by the end of seventh grade, “students should be able 
to solve problems involving percent, ratio, and rate, and extend this work to proportionality” (p. 20). 

The U.S. Department of Education’s National Center for Educational Evaluation and 
Regional Assistance (NCEE) — within the Institute of Education Sciences — initiated the Middle 
School Mathematics Professional Development Impact Study to test the impact of a professional 
development (PD) program for teachers that was designed to address the problem of low student 
achievement in topics in rational numbers. 2 The study focuses on seventh grade, the culminating 
year for teaching those topics. The study is being conducted by the American Institutes for Research 
(AIR) and MDRC together with their evaluation partners REDA International and Westat. 

Currently, through the Elementary and Secondary Education Act, the federal government 
provides significant resources for PD, but little rigorous evidence is available on the impact of PD 
on teacher and student outcomes. 3 Hundreds of studies have addressed the topic of teacher learning 
and PD (for reviews, see Borko 2004; Clewell, Campbell, and Perlman 2004; Kennedy 1998; 
Richardson and Placier 2001; Supovitz 2001; Yoon, Duncan, Lee, Scarloss, and Shapley 2007). 4 The 
most recent review of studies of the impact of teacher PD on student achievement revealed a total 
of nine studies that have rigorous designs — randomized control trials (RCTs) or certain quasi- 
experimental designs (QEDs) — that allow causal inferences to be made (Yoon et al. 2007). Four of 



2 The professional development focused on positive rational numbers. The decision to restrict the focus to positive rational numbers 
was based on advice from the study’s external advisors, who suggested that including negative rational numbers would broaden the 
scope of the content beyond what could be addressed in the allotted time for the PD program. 

3 In the 2001 reauthorization of the Elementary and Secondary Education Act of 1965 (ESEA), the Congress expanded the federal 
resources available for teacher professional development by establishing — under Title II, Part A — the Improving Teacher Quality 
State Grants program. The grants program provides support for activities designed to ensure an adequate supply of knowledgeable 
teachers, and states and school districts spent $529 million of Title II, Part A funds on teacher professional development, according to 
an analysis of spending for the 2004-2005 school year. An even more widely used source of funds for teacher professional 
development is Title I, through which states and districts spent $988 million for teacher professional development in 2004-2005 
(Birman et al 2007, p. 69). ESEA requires that schools that have been identified for improvement spend at least 10 percent of their 
Title I allocations on professional development (Title I, Part A, Section 1 116(b) (3) (A) (i)). 

4 For example, Yoon et al. (2007) alone identified 1,343 studies of PD. 
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the nine studies focused on the effect of a PD program on mathematics achievement, and none 
focused on mathematics at the middle school level. 

The Middle School Mathematics PD Impact Study is the first rigorous test of the impact of 
a PD program focused on teachers of middle school mathematics. Within 12 participating school 
districts, the study randomly assigned 77 mid- and high-poverty schools to treatment and control 
conditions and collected outcome data on teachers and students. The PD was delivered by two 
provider organizations, each of which served the treatment schools in six of the 12 participating 
districts. Seventh-grade teachers in the treatment schools had the opportunity to receive the PD 
program offered by the study and could also continue to participate in the PD activities that they 
would have received in the absence of the study. Seventh-grade teachers in the control schools 
received only the PD that they would have received in the absence of the study. 

The study has three central research questions: 

1 . What impact did the PD program provided in this study have on teacher knowledge of 
rational number topics? 

2. What impact did the PD program provided in this study have on teacher instructional 
practices? 

3. What impact did the PD program provided in this study have on student achievement in 
rational number topics? 

The study produced the following results: 

• The study’s PD program was implemented as intended. The PD providers 
delivered an average of 67.6 hours of PD per site, compared to 68 hours intended, and 
the treatment group teachers attended an average of 83 percent of the PD that was 
delivered. In surveys given to treatment and control group teachers, treatment group 
teachers reported participating in 55.4 hours more mathematics-related PD than the 
control group teachers. 

• The PD program did not produce a statistically significant impact on teacher 
knowledge of rational numbers (effect size = 0.19, p-value = 0.15). On average, 54.7 
percent of teachers in the treatment group answered test items of average difficulty 
correctly, compared with 50.1 percent for teachers in the control group. 

• The PD program had a statistically significant impact on the frequency with 
which teachers engaged in activities that elicited student thinking, one of the 
three measures of instructional practice used in the study (effect size = 0.48). This 
measure encompasses such behaviors as asking other students whether they agree or 
disagree with a particular student’s response and also includes behaviors elicited from the 
students such as offering additional justifications or strategies. Treatment teachers on 
average engaged in 1 .03 more activities per hour that elicited student thinking. The PD 
program did not produce a statistically significant impact on the other two measures of 
instructional practice: Teacher uses representations (effect size = 0.30; p-value = 0.0539) and 
Teacher focuses on mathematical reasoning (effect size = 0.19; p-value = 0.32). 

• The PD program did not produce a statistically significant impact on student 
achievement (effect size = 0.04, p-value = 0.37). 
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Overview of the PD Program 

The PD program delivered in this study was designed to develop teachers’ capability to teach 
positive rational number topics effectively. The PD program consisted of 68 contact hours, all 
addressing rational number topics, which is more PD in mathematics than most mathematics 
teachers typically receive in a single year. 5,6 The PD included a 3-day summer institute and a series of 
1-day follow-up seminars held during the school year, with in-school coaching following each 
seminar day. Within that structure, the specification of the PD program was guided by the literature, 
which is largely based on correlational research and practitioner experience. ' 

Within each topic in rational numbers, the PD program focused on two aspects of teachers’ 
content knowledge. The first, common knowledge of mathematics (CK), is the knowledge of topics 
in rational numbers that students should ideally have after completing the seventh grade. This 
knowledge includes computational or procedural skills, conceptual understanding, and problem- 
solving skills in rational number topics. 

The second aspect of teachers’ content knowledge emphasized in the PD, specialized 
knowledge of mathematics for teaching (SK), is additional knowledge of rational numbers that may 
be useful for teaching rational number topics. For example, SK includes identifying the key 
mathematical understanding within a topic or problem, identifying common errors that occur in 
student work, and choosing useful representations and explanations when teaching rational numbers. 

The summer institute and seminars blended activities intended to develop specialized 
knowledge of mathematics for teaching and to strengthen common knowledge of mathematics. The 
institutes and seminars were designed to use multiple delivery formats to provide teachers a variety 
of learning opportunities. The planned PD activities included opportunities for teachers to solve 
mathematics problems individually and in groups, make short oral presentations to explain how they 
solved problems, receive feedback on how they solved and presented their solutions, engage in 
discussions about the most common student misconceptions associated with topics in rational 
numbers, and plan lessons that they would teach during the follow-up coaching visits. 

The primary purpose of the coaching component of the PD program was to help teachers 
apply material covered in the institutes and seminars to their classroom instruction. The coaching 
component was designed to consist of 10 days of coaching provided through five 2-day visits to 
each school. During the coaching visits at each school, the facilitators focused their activities on the 
school’s seventh-grade mathematics teachers. Each 2-day coaching visit was designed to occur 
immediately after one of the 5 seminar days and to link to the preceding seminar, using both 
individual and group activities. 



5 Sixty-eight hours is the number of contact hours provided during the first year of the PD program, which is the focus of this 
report. Additional contact hours were provided in the second year of the PD program. 

6 A national survey of teachers completed in 2005—2006 found that 1 1 percent of elementary teachers and 22 percent of secondary 
teachers assigned to teach mathematics participated in professional development in mathematics lasting more than 24 hours 

(U.S. Department of Education 2009, p. 95). 

7 In the nine rigorous studies identified by Yoon et al. (2007), the variation in the features of the PD programs that were tested was 
not sufficient to draw conclusions about the characteristics of the PD programs that were effective. For example, across the nine 
studies, all PD programs were delivered in the form of a workshop or a summer institute, along with some form of follow-up 
support. 
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